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PREFACE 


This  Technical  Report  is  a  revision  and  extension  to  a  report 
previously  conducted  by  Willis  R.  Greer:  A  Method  for  Estimating 
and  Controlling  the  Cost  of  Extending  Technology  (Naval 
Postgraduate  School  Technical  Report,  NPS-54-88-002 ,  Monterey, 
CA,  March  1988)  .  Dr.  Greer's  task  was  to  develop  a  method  for 
measuring  extensions  in  the  state  of  the  art  of  technology  in 
complex  hicih  technology  systems  and  document  relationships 
between  technology  extensions  and  cost.  His  report  consisted  of 
the  following  steps: 

1.  Review  of  literature  on  costing  technology  extensions. 

2.  a.  Collection  of  data  for  18  satellite  systems  measuring  17 

distinct  technology  related  characteristics  for  each 
system. 

b.  Statistical  "reduction"  of  the  17  measures  of  technology 
characteristics  into  four  basic  dimensions  of  technology. 
Factor  analysis  was  used  and  four  factors  scores, 
representing  the  four  technology  dimensions,  resulted. 

3.  Creation  of  measures  of  technological  extension  of  new 
systems  vis-a-vis  predecessor  systems.  An  "ellipsoid" 
approach  relying  on  the  euclidean  distance  between  factor 
scores  in  4-dimensional  space  was  used. 

4.  Description  of  relationships  between  the  development  cost  of 
systems  and  the  technological  extension  required  in  creating 
the  system. 

5.  Creation  of  measures  of  variances  reflecting  the  difference 
between  expected  development  cost,  given  the  technological 
extension  required,  and  actual  development  cost. 

Further  analysis  indicates  that  the  procedures  used  in 

creating  measures  of  technological  extension  (step  3)  were 

inconsistent  with  the  way  the  basic  dimensions  of  technology  were 

measured  (step  2b)  .  More  specifically,  the  ellipsoid  approach 


li 


used  in  step  3  requires  as  its  input  dimensions  that  have  a 


"natural  zero"  point,  where  zero  implies  zero  technological 
sophistication  or  complexity.  The  factor  scores  created  in  step 
2b  do  not  possess  this  property. 

It  should  be  noted  that  Dr.  Greer's  ellipsoid  analysis 
creates  and  describes  a  novel  approach  to  measuring  technology 
extension,  one  that  is  potentially  very  useful  in  situations 
where  the  data  fulfills  the  necessary  constraints  for  its  use. 
However,  applying  his  approach  when  the  input  data  are  factor 
scores  requires  an  arbitrary  transformation  of  the  factor  scores 
which  can  only  be  justified  on  ad  hoc  basis. 

This  technical  report  first  undertakes  an  analysis  analogous 
to  Dr.  Greer's  but  uses  an  alternative  approach  to  measuring 
technology  extensions  (step  3)  .  The  analysis  continues  to 
address  the  question  of  relationships  between  technology  and 
development  cost  (step  4)  and  creates  measures  of  development 
cost  variances  (step  5) . 

Additionally,  this  technical  report  extends  the  analysis  to 
examine  the  production  cost  of  systems.  Relationships  between 
measures  of  technology  extension  and  development  cost  with 
subsequent  production  cost  are  documented.  And  models  for 
estimating  production  cost  are  presented. 

This  technical  report  is  self-contained,  in  that  it  can  be 
understood  without  reference  to  Dr.  Greer's  previous  analysis. 
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ESTIMATING  AND  CONTROLLING 
THE  COST  OF  EXTENDING  TECHNOLOGY 

ABSTRACT 

When  firms  undertake  new  development  projects,  there  is 
considerable  uncertainty  as  to  the  amount  of  cost  that  will 
eventually  be  incurred.  This  study  tests  hypotheses  concerning 
the  relationships  between  extensions  in  technology  and  costs,  and 
provides  approaches  for  estimating  and  controlling  costs.  The 
study  begins  by  examining  the  techniques  currently  available  for 
measuring  the  state-of-the-art  of  technology.  Next,  methods  for 
quantifying  the  incremental  progress  represented  by  a  particular 
project  are  reviewed  and  extended.  Third,  relationships  between 
technology  measures  and  development  time  and  development  costs 
are  formulated  and  tested.  Fourth,  variance  measures  related  to 
ucv'fei.^p.uciu.  cost  are  speciixea.  Fifth,  relationships  between  the 
scope  of  the  development  phase  of  a  program  and  subsequent 
production  ccsts  are  examined.  Finally,  the  idea  of  a 

development  cost  "premium,"  used  to  relate  development  costs  to 
production  costs,  is  introduced  and  tested.  The  workability  of 
the  approach  for  cost  prediction  and  control  is  tested  and 
demonstrated  by  using  technological  and  cost  data  from  18 
satellite  programs. 


ESTIMATING  AND  CONTROLLING 
THE  COST  OF  EXTENDING  TECHNOLOGY 


When  firms  engage  in  research  and  development  activities  they 
encounter  exceptionally  difficult  cost  control  problems.  In 
fact,  when  such  endeavors  are  undertaken  on  behalf  of  an  external 
sponsor,  a  Department  of  Defense  activity  for  example,  the 
uncertainty  involved  generally  reouires  using  some  variant  of 
cost-plus  pricing.  If  the  task  is  performed  speculatively,  or  on 
a  fixed  price  basis,  there  is  serious  cost  risk. 

Part  of  the  problem  stems  from  an  absence  of  clear  measures 
of  technology--how  "advanced"  is  a  particular  aircraft  or 
computer?  Or  how  much  more  advanced  is  the  objective  of  this 
particular  development  project?  Additionally,  there  is  no 
reliable  methodology  for  associating  development  costs  with 
different  degrees  of  technological  advance. 

This  study  examines  and  extends  techniques  currently 
available  for  measuring  the  state-of-the-art  (SOA)  of  technology 
and  technology  extensions  by  drawing  on  several  suggestions  from 
the  literature.  Relationships  of  SOA  and  SOA  extension  with 
development  time  and  development  cost  are  then  specified  and 
tested.  The  approach  produces  measures  of  variances  that  may  be 
used  in  the  process  of  controlling  development  cost. 
Relationships  between  development  cost  and  production  cost  are 
also  examined  by  introducing  and  testing  a  concept  called  the 
development  cost  "premium."  The  methodology  developed  is 
demonstrated  and  empirically  tested  with  data  from  18  satellite 
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programs.  The  end  result  is  the  establishment  of  a  series  of 
relationships  between  technology  and  costs  that  can  be  used  to 
estimate  and  control  the  cost  of  extending  technology.  We  begin 
by  reviewing  approaches  in  the  literature  to  measuring  the 
technology  embodied  in  a  system. 

MEASURING  THE  STATE  OF  THE  ART  OF  TECHNOLOGY 
The  literature  on  technology  measurement  offers  three  broad 
approaches  to  determining  the  state-of-the-art  (SOA)  of 
technology  for  a  given  set  of  related  systems.  Each  approach 
requires  the  knowledge  of  a  number  (n)  of  technology  variables 
reflecting  distinct  properties  or  characteristics.  Each  combines 
the  variables  into  a  single  SOA  measure.  For  background,  we 
review  each  approach,  discussing  advantages  and  disadvantages, 
and  conclude  that  one  approach  is  most  applicable  to  our  data. 

Judgmental  Weighting.  This  approach,  discussed  by  Gordon  and 
Munson  '19S11,  expresses  SOA  as  a  direct  combination  of  values  of 
the  technology  characteristics.  Gordon  and  Munson  suggest  two 
general  forms  of  SOA  equations. 

SOA  =  KiV-l  +  K2V2  +  .  .  .  +  Knvn  (1) 

and 

SOA  =  Vx  [K2V2  +  K3V3  +  .  .  KnVn]  (2) 

where 

Kj_  =  judgmentally  assigned  weights 

=  the  value  of  the  ith  technology-describing 
variable . 
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The  first  version  of  the  model  is  a  simple  linear  combination  of 
weighted  characteristics,  the  second  version  is  a  multiplicative 
form  intended  for  use  when  one  variable  (V^)  must  be  present  in 
the  system.  Gordon  and  Munson  provide  applications  of  the  first 
version  to  measure  the  SOA  for  computer  systems  and  the  second 
version  to  measure  the  SOA  for  antibiotic  drugs.  Similar 
approached  have  been  used  by  the  Department  of  Defense  to  measure 
aircraft  performance  FTimperlake,  et  al.,  1980]. 

While  useful  in  some  situations  the  approach  has  drawbacks. 
It  is  inherently  subjective  when  the  weights  are  assigned 
judgmentally ,  requiring  a  panel  of  experts.  It  also  presupposes 
sufficient  theoretical  understanding  of  how  individual 
characteristics  "combine"  to  produce  SOA  in  order  to  specify  ex 
ante  an  appropriate  form  of  the  model.  In  addition,  setting 
weights  becomes  quite  difficult  when  n  is  large.  Gordon  and 
Munson  suggest  that  factor  analysis  may  be  useful  to  reduce  the 
number  of  technology  descriptors,  but  the  difficulty  of  ascribing 
intuitive  meaning  to  factors  increases  the  judgmental  problem  of 
assigning  weights  when  factors  are  used  as  the  technology 
descriptors . 

Ellipsoid  fitting.  In  1985,  Dodson  [1985]  reported  an 
important  advance  in  technology  measurement  which  was  based  on 
work  he  did  much  earlier  with  Graver  [1969].  The  approach  was  to 
make  use  of  convex  (ellipsoidal)  hypersurfaces  to  represent 
particular  levels  of  technology.  In  order  to  utilize  the 
approach,  n  technology  attributes  are  specified  and  measured 
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across  a  sample  of  the  systems  under  study.  An  n-dimensional 
ellipsoid  is  then  fit  to  the  design  attribute  measurements.  The 
curve  follows  the  general  form  of  an  ellipse, 


where 

X-^  =  technology-describing  variables 

=  weights  assigned  by  the  ellipsoid  fitting  procedure. 

The  fitting  technique  uses  a  least  squares  algorithm,  and 
operates  on  proportional  distances  of  points  from  the  origin. 1 

Once  fit  to  a  sample  of  systems,  taken  from  a  common  time 
period,  the  ellipsoid  surface  can  be  viewed  as  reflecting  the  SOA 
of  technology  for  that  time  period.  The  SOA  of  an  individual 
system  can  be  determined  by  plugging  the  values  of  the  n 
technology-describing  variables  for  the  individual  system  into 
the  ellipsoid  equation  to  arrive  at  a  measure  of  the  radial 
distance  from  the  origin  for  the  system. 

The  method  can  best  be  illustrated  in  two  dimensions,  using 
data  on  aircraft  systems  from  Martino  [1985]. 2  Figure  1  plots 
indices  representing  the  speed  and  useful  load  of  eight  existing 
aircraft,  and  for  one  new  aircraft  (to  be  developed)  .  Had  the 
objective  of  the  new  aircraft  development  program  been  a  point 

-’■Programs  for  fitting  ellipsoids  are  available  in  Fortran 
and  Basic  in  Docson  and  Graver  [1969]  and  Greer  [1988], 
respectively. 

2Martino  also  reported  a  third  variable  representing 
structural  efficiency. 
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ELLIPSOID  METHOD 


Aircraft  Speed  versus  Useful  Load 
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near  the  existing  technology  surface  (represented  by  the  ellip¬ 
soid)  only  redesign  effort  would  have  been  required--no  advance 
in  prevailing  technology  would  be  necessary.  (Speed  and  useful 
load  are  attributes  that  designers  can  trade  off  against  one 
another,  subject  to  diminishing  returns) .  But  the  new  aircraft 
is  required  to  be  more  advanced  in  both  respects,  so  technology 
must  be  advanced  beyond  its  current  level. 

One  advantage  of  the  ellipsoid  approach  lies  in  the  fact  that 
the  resulting  measure  of  technology,  the  proportional,  radial 
distance  from  the  origin,  is  scale-free.  A  second  advantage  lies 
in  the  theoretical  appropriateness  of  the  ellipsoid  surface.  It 
is  consistent  with  the  constraints  designers  face  in  trading  off 
design  attributes  against  each  other  to  achieve  an  overall 
objective  (e.g.,  speed  versus  load).3  The  disadvantages  are  that 
reasonable  stability  in  the  ellipsoid  can  be  achieved  only  when 
the  number  of  variables  is  small  in  relation  to  the  sample  size,4 
and  that  the  n  measures  used  to  describe  technology  character¬ 
istics  have  to  take  on  only  positive  values  and  have  a  natural 
zero  point  so  that  the  distance  from  the  origin  is  meaningful.5 
Thus,  for  example,  factor  scores  resulting  from  variable 

3For  elaboration  of  this  last  point  see  Knight's  [1985] 
discussion  and  test  of  Grosch's  law  as  it  relates  to  design 
tradeoffs . 

4The  authors  have  found  from  experience  that  at  least  four 
observations  per  variable  are  needed. 

5More  formally,  the  measures  need  to  be  of  ratio  scale, 
rather  than  merely  interval,  ordinal  or  nominal  scale.  See 
Kerlinger  [1973,  Chapter  25].  Dodson  [1970]  discussed  other 
constraints  on  appropriate  measures  to  be  used  in  the  ellipsoid  approach. 
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reduction  techniques  such  as  factor  analysis  can  not  be 
appropriately  used  to  fit  an  ellipsoid. 

"Year-of -Techno logy"  Regression .  A  third  approach  frequently 
used  [e.q.  Alexander  and  Nelson,  1972;  Dodson,  1977]  employs 
multiple  regression  to  combine  several  technology-describing 
variables  across  a  sample  of  systems.  The  "year  of  technology" 
is  the  dependent  variable,  where  this  is  usually  interpreted  as 
the  year  in  which  the  system  became  operational. 

Y  =  a  +  b^  +  b2X2  +  .  .  .  +  bnXn  +  e  (4) 

where  Y  =  actual  year  the  system  became  operational. 

bj_  =  regression  coefficients 
X-j_  =  technology-describing  measures 
e  =  residual 

Predicted  values  (Ye)  from  the  regression  equation  for  an 
individual  system  represents  the  "year-of-technology"  or  SOA  for 
the  system,  i . e. , 

SOA j  =  Yej  =  a  +  b1X1j  +  .  .  .  bnXnj  (5) 

As  simplistic  as  this  method  seems,  related  work  by  Lienhard 
[1979]  tends  to  support  the  concept.  His  paper  studied  the  rate 
at  which  technology  is  improved,  and  how  (whether)  this  rate 
changes  through  time.  He  studied  several  forms  of  technology 
(clocks,  steam  power,  land  transportation,  low  temperatures,  air 
transportation)  over  extended  time  periods.  The  most  relevant 
observations  to  come  from  Lienhard' s  study  was  that  the  rate  of 
improvement  of  a  particular  technology,  once  established,  does 
not  change.  If  this  is  literally  correct  (and  his  data  do  seen 
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to  support  the  observation) ,  there  could  be  some  major 
implications  for  the  cost,  and  even  the  feasibility,  of 
attempting  to  effect  technological  advances  "before  their  time". 
If  a  desired  advance  could  normally  be  expected  to  occur  only  by 
some  quasi-naturally  established  date,  attempts  to  accelerate 
this  process  would  be  very  costly.  Accordingly,  the  "year  of 
technology"  approach  may  be  well  reasoned. 

MEASURING  EXTENSIONS  IN  TECHNOLOGY 

To  address  the  issue  of  advances  in  technology,  measures  of 
increments  in  technology  are  necessary.  In  general,  the 
procedure  for  measuring  increments  in  technology  is  similar 
across  the  three  approaches:  A  reference  point,  reflecting  the 
current  state  of  technology,  is  established  and  the  distance 
between  the  reference  point  and  the  technology  in  a  particular 
system  is  measured.  Out  data  (to  be  discussed  in  a  following 
section)  describes  technology  characteristics  using  factor 
scores.  Consequently  we  rejected  the  judgmental  weighting 
approach  (attaching  weights  to  factors  is  conceptually  difficult) 
and  the  ellipsoid  approach  (factor  scores  violate  the  ellipsoid 
assumptions)  and  relied  on  the  year-of-technology  approach.6 

The  essence  of  the  year-of-technology  approach  is  to  combine 
numerous  technology  descriptions  into  a  summary  technology 
measure,  expressed  in  terms  of  time  (years) .  Using  this  notion, 


6Greer  [1988]  discusses  procedures  for  creating  measures  of 
technology  extension  from  the  ellipsoid  approach. 
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we  create  three  technology  measures  for  each  individual  system: 
REACH,  ADVANCE  and  STAND. 

Recall  that  the  predicted  value  (Ye)  from  the  regression 
(Equation  5)  represent  the  year-of-technology  for  a  given  system. 
Ye  is  a  summary  measure  of  the  degree  of  technology  embodied  in 
the  system.  We  define  Ye  as  the  technological  REACH  of  the 
system. 

Alexander  and  Nelson  [1972]  and  Dodson  [1977]  argue  that  if 
the  actual  year  (Y)  a  system  was  produced  was  less  than  the 
predicted  year  (Y  <  Ye)  then  that  system  was  "ahead  of  it's 
time":  the  "state-of-the-art"  was  advanced  when  that  model  was 
developed.  (Conversely,  if  Y  >  Ye  then  the  system  was  "behind 
the  times.")  Therefore,  Ye-Y  measures  the  extent  to  which  each 
system  represented  a  technological  advance.  In  short,  we  define 
ADVANCE  as  Ye-Y 

Finally,  since  the  state  of  the  art  of  technology  at  any 
moment  is  defined  in  terms  of  years,  che  year  in  which  a  system 
becomes  operational  represents  where  technology  in  general 
currently  "stands."  Hence  STAND  simply  equals  Y. 

Our  interest  in  these  three  technology  measures  is  in  using 
them  to  assist  in  cost  prediction  and  control.  This  will  be 
illustrated  with  a  sample  of  satellite  systems. 


SAMPLE  AND  DATA 

Our  sample  consisted  of  18  satellite  programs;  with  the 
initial  satellite  of  each  program  first  launched  between  1966  and 
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1986.  The  sample  included  several  types  of  satellites  -  communi¬ 
cations,  surveillance,  navigation,  etc.  Data  were  obtained  from 
the  U.  S.  Air  Force.7 

All  the  methods  of  measuring  the  state-of-the-art  of 
technology  and  technology  extensions  require  basic  data 
describing  properties  or  characteristics  of  the  systems  under 
study.  Data  describing  84  technical  characteristics  of  each  of 
the  18  satellites  were  available. 

Variable  Selection 

In  selecting  variables  to  be  used  in  measuring  the  technology 
embedded  in  the  category  of  systems  to  be  studied,  technical 
expertise  must  be  sought.  In  selecting  variables,  the  experts 
should  bear  in  mind  that  the  technology-describing 
characteristics  must  be  at  least  partially  alterable  by 
engineering  development  decisions.  They  should  choose 
characteristics  which  are  goals  of  the  design  process.  Ideally 
the  variables  should  be  specified  so  that  increasing  values 
correspond  to  greater  technical  difficulty.  Finally,  the  values 
of  the  variables  should  be  ascertainable  during  the  early  stages 
of  the  system's  life-cycle  so  they  may  be  used  in  the  process  of 
predicting  costs. 


7These  data  were  supplied  by  Headquarters,  Space  Division 
( AFSC) ,  Los  Angeles  Air  Force  Station,  Los  Angeles,  CA.  While 
the  data  are  not  classified,  the  authors  were  asked  not  to 
identify  specific  satellites  by  name  or  designator.  To  honor 
this  request  the  systems  will  be  referred  to  only  by  randomly- 
assigned  code  letters.  The  authors  are  particularly  indebted  to 
Captain  Blain  Webber,  USAF,  for  his  assistance  and  cooperation. 
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Some  of  the  84  properties  included  in  the  data  set  could  be 
considered  design  objectives,  and  therefore  appropriate  as 
technology  indicators,  but  many  others  were  simply  byproducts  of 
the  design.  Others  were  potentially  useful,  but  were  not  stated 
in  a  form  that  revealed  much  about  the  technology  embodied. 

Our  first  step  was  to  identify  which  of  the  84 
characteristics  were  relevant  to  describing  the  state  of 
technology  in  the  satellite  systems.  Technical  expertise  was 
sought  and  extensive  data  review,  conversations  and  conferences 
took  place  among  a  small  group  of  satellite  experts.®  The  result 
of  these  conferences  was  consensus  identification  of  17  variables 
that  are  well-suited  to  describing  satellite  technology.  The  17 
variables  are  listed  in  Appendix  I. 

Second,  with  17  variables  to  describe  18  systems,  variable 
reduction  was  necessary.  Principle  components  factor  analysis 
with  varimax  orthogonal  rotation  was  conducted.9  The  result  was 
a  final  factor  matrix  explaining  81.7%  of  the  variance  in  the 
technology  variables.  Four  separate  factors  had  eigenvalues 
greater  than  one.  They  were  retained  for  use  in  the  remainder  of 
the  analysis.  The  four  factors  reflect  four  different  aspects  of 
satellite  technology  which  we  labeled  as  follows: 

®The  authors  are  indebted  to  the  Naval  Postgraduate  School, 
and  particularly  to  Dr.  Allen  E.  Fuhs,  Distinguished  Professor  of 
Aeronautics  &  Space,  Dr.  Richard  W.  Adler,  Adjunct  Professor, 
Elec.  &  Comp.  Engineering,  and  Mr.  Marty  Mosier,  Staff  Engineer, 
Space  Systems,  for  their  invaluable  assistance. 

9For  a  comprehensive  discussion  of  factor  analysis  see 
Harman  [ 1976]  . 
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Factor  1:  Mission  requirements 
Factor  2:  Orbital 
Factor  3:  Electrical  Power 
Factor  4:  Environment 

A  detailed  description  of  the  factor  analysis  is  contained  in 
Appendix  II.  Factor  scores  for  each  of  the  18  satellites  are 
contained  in  Table  1. 

CREATING  TECHNOLOGY  MEASURES 

Following  the  year-of-technology  approach,  the  launch  year 
(cf  the  first  unit)  of  each  satellite  program  was  regressed 
against  the  MISSION,  ORBITAL,  ELECTRICAL  and  ENVIRONMENT  factor 
scores.  Results  are  in  Table  2.  Note  that  the  overall  model  is 
significant  and  explains  a  large  portion  of  the  variance 
(adjusted  R2  =  .65).  In  addition,  each  factor  is  reasonably 
significant10  with  positive  coefficients,  consistent  with 
increasing  values  of  the  four  technology  descriptors  reflecting 
increasing  technology  over  time.  This  provides  some  confirmation 
that  the  factor  scores  appropriately  reflect  dimensions  of 
technology  and  technology  growth. 

Figure  2  shows  a  plot  of  year-of-launch  (Y)  versus  year-of- 
technology  (Ye)  .  Using  the  approach  outlined  earlier,  we  can 
determine  values  for  STAND,  REACH  and  ADVANCE  for  each  system. 
By  way  of  illustration  we  might  view  system  H,  highlighted  in  the 
Figure.  It  was  actually  launched  in  1969  (STAND)  but  has  a  year- 

10Since  our  hypotheses  are  directional  we  use  one  tail  tests 
of  significance. 
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TABLE  1 
Factor  Scores 


em  Fact  1  Fact  2  Fact  3  Fact  4 


0.5481 

1.4894 

1.4514 

1.3038 

-0.7021 

-0.6149 

0.8253 

-0.8771 

1.4523 

-1.1906 

0.3233 

2.3646 

-0.2591 

1.1941 

0.8042 

0.0857 

-0.8919 

-0.7342 

0.4668 

-0.6761 

-0.5195 

0.6648 

-1.8337 

0.8363 

-0.2870 

-1.3133 

-1.7576 

0.8937 

0.4619 

-0.8923 

-0.4650 

-1.1932 

2.1776 

-1.0966 

0.4696 

-0.3093 

-1.4171 

-1.0918 

1.5999 

-0.0233 

-1.0725 

0.9872 

0.2053 

0.4739 

-0.9641 

-0.6474 

0.0670 

-0.2184 

-0.1629 

-0.8299 

0.2157 

-0.4064 

-1.1595 

0.7442 

-1.0434 

1.0676 

0.6814 

0.9718 

-0.4915 

-1.6992 

0.1755 

0.7231 

-0.4819 

-0.8667 

0.7529 

0.4861 

-1.7235 

-0.4232 

1.1859 

1.1505 

0.8674 

-0.3126 

TABLE  2 


YEAR-OF-TECHNOLOGY  REGRESSION 


Dependent  Variable:  Launch  Year 


Independent 

Variables 

Coefficients 

t 

Sianif icance 

Intercept 

75.720 

— 

— 

MISSION 

2.213 

2 . 84 

.007 

ORBITAL 

1.103 

1.41 

.  090 

ELECTRICAL 

3 . 698 

4 . 74 

.  001 

ENVIRONMENT 

1.349 

1.73 

.  053 

Model  Statistics 
F  :  8.85 

Significance:  .0011 

R2  :  .73 

Adjusted  R2  :  .65 


*One  Tailed  Tests 
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FIGURE  2 

YEAR  OF  TECHNOLOGY  PLOT 

YEAR  OF  TECHNOLOGY 


3  65  70  75  80  85 


LAUNCH  YEAR 


of-technology  value  (REACH)  that  falls  between  1972  and  1973. 
Thus  it  was  three  to  four  years  ahead  (ADVANCE)  of  the  general 
trend  in  technology.  More  precise  measures  for  system  H  and  the 
other  systems  are  in  Table  3. 

DEVELOPMENT  COSTS  -  HYPOTHESIS  AND  TESTS 

Our  next  objective  was  to  explain  development  costs  using  the 
technology  measures.  Data  relevant  to  this  (and  later)  portions 
of  the  analysis  are  in  Table  4 .  The  reported  development  cost 
figures  are  "nonrecurring"  costs  provided  by  the  Air  Force  for 
each  system,  adjusted  to  constant  1986  dollars  (thousands) .  The 
development  time  figures  report  the  time  elapsed  in  months  from 
the  awarding  of  the  development  contract  to  the  first  launch  of 
the  satellite.  The  reported  production  cost  figures  are 
"recurring"  costs  per  unit,  i.e.,  costs  incurred  in  manufacturing 
each  unit  of  a  given  satellite  program  after  the  development 
phase,  also  adjusted  to  1986  dollars.  The  production  run  figures 
are  the  number  of  units  produced  for  each  separate  satellite 
program . 

System  A  was  excluded  from  Table  4  because  it  was  known  to 
have  been  a  very  minor  upgrade  of  another  satellite  for  which 
data  were  not  available.  Data  on  development  time  for  systems  D 
and  F  were  unavailable  because  the  contract  dates  were  unknown. 

D  and  F  were  excluded  from  analyses  requiring  development  time 
and  production  costs.  The  data  were  complete  for  the  remaining 
15  systems,  except  that  only  the  year  of  the  contract  award,  not 
the  month,  was  known  for  systems  R,  C  and  I.  The  working 
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TABLE  3 


TECHNOLOGY  MEASURES 


SYSTEM 

STAND 

( YEAR  OF  LAUNCH) 

ADVANCE 

REACH 

f YEAR  OF  TECHNOLOGY) 

A 

86 

-.30 

85.70 

B 

76 

-.64 

75.36 

C 

82 

.  01 

82.01 

D 

78 

1.55 

79.55 

E 

77 

-3 .25 

73.75 

F 

66 

5.50 

71.50 

G 

66 

2.34 

68.3  4 

H 

69 

3 .43 

72 .43 

I 

83 

-2.35 

80.65 

J 

75 

2.26 

77.26 

K 

79 

-3 . 16 

75 . 84 

L 

73 

-  .  17 

72 .83 

M 

76 

-1.30 

74.70 

N 

74 

-2 .44 

71.56 

0 

71 

3 . 19 

74 . 19 

P 

76 

-2 . 05 

73.95 

Q 

76 

-5.02 

70.98 

R 

80 

2 .40 

82 .40 

Ua 


TABLE  4 


DEVELOPMENT  AND  PRODUCTION  DATA 


DEVELOPMENT 

DEVELOPMENT 

PRODUCTION 

PRODUCTION 

SYSTEM 

COST 

TIME  (months) 

COST 

RUN 

A 

_ 

_ 

- 

B 

91707 

37 

23483 

4 

C 

157820 

64 

46865 

2 

D 

50393 

- 

- 

3 

E 

108085 

37 

54094 

3 

F 

509262 

- 

— 

5 

G 

273270 

20 

49847 

23 

H 

73594 

25 

50414 

1 

I 

451274 

86 

51972 

6 

J 

155524 

46 

118752 

1 

K 

14943 

36 

17697 

1 

L 

37228 

32 

7715 

3 

M 

319499 

37 

387034 

1 

N 

32585 

28 

11266 

3 

0 

116580 

27 

35666 

4 

P 

121931 

33 

56574 

3 

Q 

64384 

37 

28048 

3 

R 

180653 

56 

39432 

15 

assumption  adopted  was  to  date  the  contract  award  at  mid-year, 
but  other  assumptions  were  analyzed  for  result  sensitivity  with 
very  little  disparity. 

Development  Cost  and  Technological  Complexity 

Our  first  hypothesis  was  that  there  is  a  direct  relationship 
between  development  cost  and  the  scope  of  the  development  task  at 
hand,  measured  in  terms  of  technological  complexity.  Two 
variables  capture  complexity:  STAND  measures  the  current  state 
of  technology  at  the  time  of  project  development.  As  the  state 
of  technology  increases,  designers  are  operating  at  a  greater 
level  of  complexity  and  consequently  costs  of  extending 
technology  are  hypothesized  to  increase.  ADVANCE  measures  the 
increment  in  technology  to  be  achieved  by  the  development 
project;  costs  are  also  hypothesized  to  increase  with  magnitude 
of  the  advance  required.  (REACH  is  simply  the  sum  of  STAND  and 
ADVANCE;  consequently  it  contains  no  additional  information  and 
is  excluded  from  the  analysis) . 

H]_:  Development  Cost  =  f  (+  STAND,  +  ADVANCE) 

To  test  the  first  hypothesis,  a  multiple  regression  was  run. 
Results  are  in  Table  5.  Coefficients  for  both  STAND  and  ADVANCE 
are  positive  as  expected,  but  neither  is  significant  at 
traditional  levels.  The  model  as  a  whole  is  unimpressive:  The 
R2  is  low  (.12)  and  the  overall  model  insignificant. 

The  low  explanatory  ability  of  the  model  is  not  totally 
unexpected.  Dodson  [1977]  also  attempted  to  predict  development 
costs  (of  computer  systems)  using  technology  variables,  with 
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TABLE  5 


REGRESSION  OF  DEVELOPMENT  COST  ON  TECHNOLOGY  MEASURES 


Dependent  Variable:  Development  Cost 


Independent 

Variables  Coef f icients  t  Significance* 


Intercept 

STAND 

ADVANCE 


30937 

1749  .19 

18833  1.23 


.425 
.  120 


Model  Statistics 
F  :  .95 

Significance:  .4120 

R2  :  .12 

Adjusted  R2  :  -.01 


*One  Tailed  Tests 


13a 


s 


little  success.  He  concluded  that  his  poor  result  was  due  to 
incomplete  modeling. 

Development  Time  and  Technological  Complexity 

We  hypothesize  that  there  is  an  intervening  variable  between 
development  cost  and  technological  complexity:  development  time. 
Complexity  affects  time;  time  effects  cost.  Our  second 
hypothesis  then  is  that  the  difficulty  of  the  development  task, 
as  measured  by  the  time  required  for  its  completion,  is  a 
positive  function  of  technological  complexity. 

H2 :  Development  Time  =  f  (+  STAND,  +  ADVANCE) 

Table  6  shows  results  of  regressing  development  time  on  the 
technology  variables.  Results  are  considerably  more  impressive. 
The  model  is  highly  significant  with  an  adjusted  R2  of  .75.  Both 
STAND  and  ADVANCE  have  significant  positive  coefficients  as 
expected.  The  actual  time  required  to  complete  a  development 
project  is  a  function  of  both  the  level  of  technological 
complexity  at  which  the  task  is  taking  place  and  the  increment  in 
technology  to  be  achieved. 

Development  Cost  and  Development  Time 

Development  cost  should  be  related  to  development  time,  but 
development  cost  may  not  be  a  smooth  function  of  time.  If  a 
program  drags  on  beyond  its  intended  completion  date,  additional 
resources  may  be  employed  to  accelerate  its  completion.  Hence, 
it  may  be  relatively  more  costly  to  compress  the  required 
accomplishment  into  an  increasingly  abbreviated  time  horizon.  We 
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TABLE  6 


TIME  REGRESSION 


Dependent  Variable:  Development  Time  (months) 


Independent 

Variables  Coefficients  _ t 


Intercept 

STAND 

ADVANCE 


-239 

3.70 

2.03 


6.55 

2.07 


Model  Statistics 
F  :  22.25 

Significance:  .0001 

R2  :  .77 

Adjusted  R2  :  .75 


*One  Tailed  Tests 


ianif icance* 

.  001 
.030 
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took  the  predicted  tiroes  from  the  above  model  as  a  "natural" 
development  time,  and  the  residuals  as  departures  (which  may  or 
may  not  have  been  planned  ex  ante)  from  this  natural  time.  We 
expect  both  time  measures  to  be  positively  associated  with  cost, 
suggesting  the  following  hypothesis: 

H3 :  Development  Cost  =  f  (+  Predicted  Time,  +  Residual  Time) 
where  Residual  Time  =  Actual  Time  -  Predicted  Time. 

The  results  of  the  multiple  regression  (Table  7)  are  good, 
although  not  as  strong  as  the  previous  test.  Both  time  variables 
have  reasonably  significant  positive  coefficients  as 
hypothesized,  and  as  expected  the  coefficient  for  the  residual 
time  term  is  larger  than  that  for  predicted  time.  This  is 
consistent  with  extension  of  a  project  beyond  its  natural 
development  time  leading  to  increased  time  pressure,  increased 
resources  expended  to  accelerate  completion  and  increased  costs. 
Additionally  note  that  the  explanatory  anility  of  this  model 
(adjusted  R2 )  has  improved  markedly  relative  to  the  first  model 
which  explained  cost  directly  by  the  technology  variables, 
indicating  that  development  time  is  indeed  an  intervening 
variable  in  the  determination  of  costs. 

INTERPRETATION  OF  RESULTS  FOR  CONTROL 

The  most  basic  principle  of  cost  control  is  to  attribute 
differences  between  anticipated  costs  and  actual  costs  to  causes. 
That  is,  to  explain  variances.  The  preceding  analysis  affords  an 
opportunity  to  do  that.  Consider  the  information  contained  in 


TABLE  7 


DEVELOPMENT  COST  REGRESSION 


Dependent  Variable:  Development  Cost 
Independent 

Variables  Coefficients  _ t 


Intercept 
Predicted  Time 
Residual  Time 


52841 

2350 

10482 


1.55 

3.59 


Model  Statistics 
F  :  7.64 

Significance:  .0073 

R2  :  .56 

Adjusted  R2  :  .49 


*0ne  Tailed  Tests 


Significance* 


.  074 
.  002 
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Table  8. 


The  column  labeled  "Ex  Ante  Cost  Estimate"  was 


constructed  in  the  following  way: 

1.  The  particular  system's  values  for  STAND  and  ADVANCE 
(both  determinable  ex  ante)  were  entered  in  the  "Time"  regression 
to  predict  a  time  that  would  be  required  for  the  system's 
development . 

2.  To  calculate  the  ex  ante  prediction  of  development  cost, 
the  Predicted  Time  was  input  to  the  "Cost"  regression,  with 
Residual  set  equal  to  zero  (thereby  assuming  the  predicted  time 
would  be  achieved) . 

Next  the  actual  time  for  the  project  was  compared  with 
the  predicted  time  to  determine  the  Residual  (ex_Rost) •  The  Cost 
regression  was  then  revisited  with  values  for  both  variables,  and 
a  new  cost  estimate  constructed,  considering  the  actual  time  for 
the  project.  This  produced  the  column  titled  "Cost  Est  Based  on 
Act  Time". 

The  difference  between  the  ex  ante  cost  estimate  and  the 
cost  estimate  based  on  the  project's  actual  time  has  been  termed 
the  "Variance  Due  to  Time".  This  figure  is  a  best  estimate  of 
the  portion  of  the  total  variance  that  can  be  attributed  to  the 
cost  consequences  of  time  delays  (or  to  fortuitus  and  perhaps 
unforeseen  acceleration  of  the  schedule)  .  Negative  figures  are 
favorable,  positive  are  unfavorable. 

When  Actual  Cost  is  compared  with  the  cost  estimate  based  on 
actual  time  the  result  is  a  "Cost  Control  Variance".  Given  that 
the  project  actually  took  t  months  to  complete,  the  cost  should 
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TABLE  8 


DEVELOPMENT  COST  VARIANCE  DATA 


SYSTEM 

EX  ANTE 
COST 

ESTIMATE 

VARIANCE 
DUE  TO 
TIME 

COST  EST. 
BASED  ON 
ACT.  TIME 

COST 

CONTROL 

VARIANCE 

ACTUAL 

COST 

B 

150150 

-46173 

103977 

-12270 

91707 

C 

205478 

-9927 

195551 

-37731 

157820 

E 

146449 

-29664 

116784 

-8699 

108085 

G 

77307 

100530 

177837 

101433 

279270 

H 

108593 

13394 

121987 

-48393 

73594 

I 

202957 

231936 

434894 

16380 

451274 

J 

155288 

25253 

180541 

-25017 

155524 

K 

164271 

-119643 

44628 

-29685 

14943 

L 

126265 

7948 

134214 

-96985 

37228 

M 

147029 

-32254 

114775 

204724 

319499 

N 

124175 

-24658 

99517 

-66931 

32585 

0 

124871 

-38247 

86624 

29956 

116580 

P 

143478 

-58343 

85134 

36797 

121931 

Q 

129307 

46796 

176103 

-111718 

64384 

R 

199461 

-66948 

132513 

48140 

180653 

It  a 


TOTAL 

VARIANCE 

-58442 

-47658 

-38364 

201964 

-34999 

248317 

236 

-149328 

-89037 

172469 

-91590 

-8291 

-21546 

-64922 

-18808 


have  been  "Cost  Est  Based  on  Act  Time".  The  actual  cost  was  a 
different  amount,  so  the  variation  is  attributed  to  the 
contractor's  success  (or  lack  thereof)  in  cost  control.  Again, 
negative  figures  are  favorable. 

The  variances  have  been  shown  graphically  in  Figure  3.  The 
variances  are  expressed  as  a  percentage  of  Ex  Ante  Cost.11 

As  Figure  3  indicates,  there  were  non-trivial  unfavorable 
total  variances  for  three  of  the  15  programs(G,  I,  M)  .  For 
program  I  the  cause  of  the  unfavorable  variance  appears  to  have 
been  mostly  attributable  to  a  timing  problem.  For  Program  M  the 
origin  of  the  problem  was  cost  control.  Both  cost  control  and 
timing  problems  appear  to  have  been  significant  contributors  to 
the  total  variance  for  program  G;  its  total  variance  is  by  far 
the  largest.  (If  these  variances  seem  large  compared  with  what 
the  reader  is  accustomed  to,  consider  the  fact  that  there  is  far 
greater  uncertainty  surrounding  a  research  project's  costs  than 
those  of  a  routine  manufacturing  operation.) 

Program  J  is  interesting  in  that  an  unfavorable  cost  control 
variance  is  offset  by  successful  control  over  time,  leaving  the 
total  cost  variance  insignificantly  small. 


i:iThe  variances  could  also  have  been  expressed  as  a 
percentage  of  Est.  Cost  Based  on  Actual  Time  or  Actual  Cost.  Use 
of  either  of  these  alternatives  would  of  course  change  the 
individual  percentages,  but  not  the  sign  of  the  variance  nor  the 
relative  size  of  the  time,  cost  control  or  total  variance  for  an 
individual  system. 
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FIGURE  3 

DEVELOPMENT  COST  VARIANCES 


The  most  significant  favorable  variances  are  for  programs  K, 
L  and  N.  Successful  cost  control  is  the  primary  source  of  the 
favorable  variance  for  L  and  N.  Program  K  has  the  largest 
favorable  variance  and  while  both  cost  control  and  time  control 
contribute,  control  over  time  is  the  primary  explanation. 

ANALYSIS  OF  PRODUCTION  COST 

With  completion  of  the  development  phase,  three  kinds  of 
information,  reflecting  technological  complexity,  development 
stage  costs  and  cost  variances,  are  available.  Our  next  question 
was  whether  our  measures  resulting  from  the  analysis  of  the 
development  phase  are  useful  in  explaining  production  costs. 

In  the  following  sections  we  develop  and  test  four 
hypotheses  concerning  the  relationship  of  production  cost  with 
the  "scale"  of  the  development  program,  cost  control  during  the 
development  phase,  the  technological  complexity  of  the  system, 
and  the  production  run. 

Production  Cost  and  the  Scale  of  the  Development  Phase 

To  summarize  the  previous  analysis:  We  first  used  measures 
of  the  technological  complexity  of  the  program  (ADVANCE,  STAND) 
to  predict  development  time.  We  calculated  Ex  Ante  Development 
Cost  directly  from  predicted  development  time.  (One  is  a  linear 
transformation  of  the  other;  hence  they  both  capture  the  same 
variance  and  are  equivalent  when  used  in  linear  models  such  as 
regression.)  Ex  Ante  Development  Cost  is  our  best  summary 
measure  of  the  scale  of  the  development  program  known  prior  to 
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its  start.  We  then  analyzed  the  differences  between  Ex  Ante  and 
Actual  Development  Cost  into  variances.  The  Ex  Ante  Cost  plus 
the  variances  equal  the  Actual  Cost,  which  is  our  best  summary 
measure  of  the  scale  of  the  development  program  known  after  its 
completion . 

Our  initial  hypothesis  is  that  production  costs  are 
positively  related  to  the  scale  of  the  development  program; 
simply  put,  systems  that  are  costly  to  develop  are  costly  to 
produce . 

H4 :  Production  Cost  =  f  (+  scale  of  the  development 

project) 

If  the  variances  due  to  time  or  cost  control  are  isolated  to  the 
development  phase  of  a  program  and  have  no  implications  for 
production  cost,  then  Ex  Ante  Development  Cost  should  provide  the 
best  measure  of  the  scale  of  the  program  to  use  in  explaining 
production  cost.  Alternatively,  if  variances  during  the 
development  phase  indicate  conditions  that  can  be  expected  to 
recur  during  the  production  phase  of  the  program,  then  the  ex 
post  Actual  Development  Cost  should  be  a  better  measure  of  scale 
to  explain  production  cost.  Our  intuition  tells  us  that  cost 
overruns  during  development  are  associated  with  higher  cost 
during  production. 

It  is  necessary  to  control  for  another  factor  potentially 
affecting  cost.  The  production  cost  measures  are  average  unit 
costs  determined  by  dividing  the  total  production  cost  for  a 
satellite  program  by  the  number  of  units  produced  (cost  per 
specific  individual  unit  were  not  available) .  It  is  well 
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established  that  unit  production  cost  typically  decreases  as  the 
number  of  units  produced  increases,  due  to  the  learning  curve 
effect.  We  control  for  this  effect  by  explicitly  testing  the 
following: 

H5:  Production  Cost  =  f  (-  Production  Run) 

where  the  Production  Run  is  the  number  of  units  produced  for  each 
satellite  program. 

Table  9  (Models  1  and  2)  shows  separate  regressions  of 
Production  Cost  on  the  two  alternative  development  cost  measures: 
Ex  Ante  and  Actual  Development  Costs,  respectively.  Production 
Run  is  included  in  each  model  as  a  control  and  as  a  test  of 
Hypothesis  5.  The  natural  log  of  each  of  the  variables  is  used 
in  the  regressions,  for  two  reasons. 

First,  learning  curve  theory  states  that  the  decrease  in 
production  cost  with  increases  in  production  quantity  j  *•  log 
linear  rather  than  linear.  Hence  logged  variables  are  the 
theoretically  appropriate  measures  when  used  in  linear 
regression .  •J-2 

Second,  the  regressions  of  production  cost  on  development 
cost  are  considerably  heteroscedatic,  i.e.,  residuals  are  larger 
when  cost  is  higher.  This  violates  an  assumption  of  regression 
that  error  variance  is  constant  over  all  observations,  resulting 
in  residuals  that  are  not  of  minimum  variance.  A  common  solution 
to  this  problem  is  to  log  the  variables  (see  Neter  and  Wasserman, 

12For  background  on  learning  curve  theory  see  Kaplan  [1982, 
pp .  97-105]  and  Womer  [1979]. 
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TABLE  9 


PRODUCTION  COST  REGRESSIONS 


Dependent  Variable: 

Production 

Costs  (log) 

MODEL  1 

Independent 

Variables  Coefficient 

t 

Sicmif  icance* 

Intercept 

7.219 

_ 

ExAnte  Dev.  Cost (log) 

.304 

.29 

.  388 

Production  Run (log) 

-.202 

-.72 

.244 

Model  Statistics 

F  :  .35 

R2  : 

.  05 

Significance  :  .7125 

Adj .  R2  : 

-.10 

MODEL  2 

Independent 

Variables  Coefficient 

t 

S ianif icance* 

Intercept 

.  041 

Act.  Dev.  Cost (log) 

.969 

7 . 02 

.  0001 

Production  Run (log) 

-.576 

-4 .31 

.  0005 

Model  Statistics 

F  :  26.17 

R2  : 

.81 

Significance  :  .0001 

Adj .  R2  : 

.78 

MODEL  3 

Independent 

Variables  Coefficient 

t 

Significance* 

Intercept 

6.72 

_ 

_ 

ExAnte  Dev.  Cost (log) 

.369 

.  65 

.  265 

Time  Variance 

3 . 3xl0-6 

1.79 

.  052 

Cost  Control  Variance 

9.7X10-6 

5.33 

.  001 

Production  Run  (log) 

-.455 

-2.74 

.  010 

Model  Statistics 

F:  8  20 

R2  : 

.77 

Significance:  .0034 

Adj .  R2 : 

.  67 

*  One  Tailed  Tests 


-  a 


1974).  Hence  logged  variables  are  methodologically  more 
appropriate . 

Results  in  Table  9  show  that  Model  2  is  clearly  superior  to 
Model  1  with  respect  to  significance  and  explanatory  power 
(adjusted  R2  of  .78).  Production  cost  is  strongly  associated 
with  the  scale  of  the  development  program,  measured  ex  post. 
Further  more,  when  development  scale  is  "properly”  measured  ex 
post,  production  cost  is  also  seen  to  be  significantly  dependent 
on  the  production  run.  Longer  runs  reduce  per  unit  cost. 

Comparing  the  results  of  the  two  models,  the  implicit 
inference  from  the  poor  explanatory  ability  of  Ex  Ante 
Development  Cost  is  that  variances  incurred  in  the  development 
phase  of  a  program  are  "permanent"  in  that  they  contain 
information  beyond  that  reflected  in  Ex  Ante  Development  Cost 
relevant  to  explaining  subsequent  production  costs.  Stated  as  a 
hypothesis : 

H6:  Production  Cost,  unexplained  =  f  (+Dev.  Time  Variance, 

by  ex  ante  development  cost  +Dev.  Cost  Control  Variance) 

We  test  this  explicitly  by  additionally  including  the  two 


development 

cost  variances  in 

Model 

1 

and  re-running 

the 

regression. 

Results  (Model  3, 

Table 

9) 

show  both  the 

time 

variance  and  cost  control  variance  incurred  during  development  to 
be  strongly  associated  with  subsequent  production  cost. 
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Production  Cost  and  Technological  Complexity 

Recall  that  Ex  Ante  Development  Cost  is  a  direct  linear 
combination  of  two  technology  variables  (ADVANCE  and  STAND)  and 
that  Ex  Ante  Development  Cost  had  little  ability  to  explain 
production  cost.  This  might  suggest  that  technology  is 
unimportant  in  explaining  production  cost.  Such  a  conclusion  may 
be  premature . 

We  know  that  development  costs  (ex  post)  are  a  good 
predictor  of  production  costs;  that  the  scale  effect  holds.  But 
the  degree  to  which  development  costs  are  associated  with 
production  costs  may  depend  on  the  technological  complexity  of 
the  system  produced. 

The  outcome  of  a  development  project  is  the  creation  of  the 
first  unit  or  prototype  of  a  particular  system.  The  cost  of  that 
prototype  system  will  include  both  "pure"  development  cost 
associated  with  extending  technology  and,  additionally,  the 
manufacturing  costs  (materials,  labor,  overhead)  associated  with 
construction.  The  pure  development  costs  are  non-recurring.  The 
construction  costs  recur  with  the  production  of  each  additional 
unit  after  the  prototype. 13  Let's  construct  a  ratio,  labeled  the 
"development  premium"  ratio,  to  relate  prototype  costs 
(numerator)  with  the  costs  of  additional  units  (denominator) . 


13In  fact  that  is  how  development  and  production  costs  are 
defined  in  this  study,  in  terms  of  ncn-recurring  and  recurring 
costs . 
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Development  Cost  +  Production  Cost 
Development  Premium  Ratio  =  _ 

Production  Cost 

Note  this  is  just  a  way  of  relating  development  and  production 
costs  while  controlling  for  scale. 

Assume  for  a  moment  a  "development"  program  that  requires 
"zero"  extension  of  technology  or  is  of  "zero"  technological 
difficulty.  In  essence  "pure"  development  cost  should  be  zero 
and  the  cost  of  the  prototype  should  equal  the  cost  of  producing 
any  follow-on  unit.  The  development  premium  ratio  should  equal 
1;  there  should  be  no  "premium"  to  extend  technology  by  zero 
amount.  Alternatively  assume  a  development  project  with  "great" 
technological  complexity.  The  pure  development  costs  should  be 
large  and  should  not  recur  for  follow-on  units;  thus  the 
development  cost  "premium"  should  be  large  (and  the  ratio  greater 
than  1)  .  If  the  size  of  the  development  cost  premium  is  driven 
by  the  technological  complexity  of  the  task  then  the  ratio  should 
be  predictable  using  a  measure  of  the  technological  complexity  of 
the  system  being  developed  and  produced.  REACH  is  a  measure  of 
the  total  technological  complexity  of  the  systems.  Operationally 
our  hypothesis  is: 

H7 :  Development  Premium  =  f  (+  REACH) 

Table  10  shows  results  of  regressing  the  premium  ratio  on  REACH. 
(Production  Run  is  again  included  as  a  control.  Production  Run 


should  have  a  positive  sign  in  this  regression  since  longer 
production  runs  imply  smaller  unit  production  cost,  reducing  the 


TABLE  10 


DEVELOPMENT  COST  PREMIUM  REGRESSION 


Dependent  Variable:  Development  Premium  Ratio 


Independent 

Variables  Coefficients  t  Significance* 


Intercept 

REACH 

Production  Run (log) 


-7.750 
.  135 
1.647 


1.42  .091 

4.03  .001 


Model  Statistics 


F:  9.01 

Significance:  .0041 

R2:  .60 

Adjusted  R2 :  .53 


*One  Tailed  Tests 


denominator  proportionately  more  than  the  numerator,  and  thus 
increasing  the  ratio.)  The  coefficient  for  REACH  is  positive  as 


expected  and  approaches  traditional  levels  of  significance.  The 
implication  is  that  the  overall  technological  complexity  of  a 
system  is  useful  in  explaining  the  degree  to  which  development 
cost  may  exceed  production  cost.14  The  adjusted  R2  of  this 
regression  is  not  as  large  as  that  found  in  Model  2,  Table  9,  but 
use  of  the  ratio  has  already  controlled  for  scale,  so  this 
regression  explains  only  the  additional  variance  not  explainable 
by  scale. 

Production  Cost  Variance 

The  regression  model  in  Table  10  provides  an  equation  for 
estimating  production  costs  which  includes  three  variables 
(Actual  Development  Cost,  Production  Run,  REACH) . 

Dev.  Cost  +  Prod.  Cost 

_  =  a  +  bj  REACH  +  b2  Prod.  Run 

Prod.  Cost 

The  equation  captures  a)  the  scale  of  the  development  phase 
(through  the  Development  Cost  measure) ,  b)  the  effect  of  learning 
on  production  cost  (through  the  Production  Run  measure)  and 
c)  the  development  cost  premium  idea  (through  REACH) .  REACH  is 
known  prior  to  development.  Development  Cost  is  known  after 

14  We  ran  a  regression  using  STAND  and  ADVANCE  in  place  of 
REACH.  The  coefficient  for  STAND  was  significant  (p  =  .094) 

while  that  for  ADVANCE  was  not.  The  adjusted  R2  of  the 
regression  was  slightly  lower  (.50),  suggesting  that  REACH  alone 
is  our  best  technology  measure  for  capturing  the  relationship 
between  development  and  production  cost. 


development  but  prior  to  production.  The  actual  production  run 
is  known  only  following  production,  but  is  a  variable  a  cost 
analyst  can  set  prior  to  production  in  order  to  estimate  unit 
costs  at  an  anticipated  production  level. 

By  plugging  in  (known)  values  for  the  three  variables  and 
solving  for  production  cost,  we  arrive  at  a  predicted  production 
cost.  Subtracting  from  actual  production  cost  provides  a  measure 
of  production  cost  variance  (positive  values  are  unfavorable) . 
While  we  do  not  have  the  ability  to  break  the  variance  down  into 
causes,  the  model  does  provide  a  benchmark  for  assessing  if 
production  cost  is  out  of  line,  given  the  knowledge  gained  during 
the  development  phase. 

Figure  4  plots  the  production  cost  variance  as  a  percentage 
of  the  production  cost.  While  some  of  the  variances  are;  not 
"small",  overall  they  are  considerably  smaller  than  the 
development  cost  variances  displayed  previously  in  Figure  3. 
This  is  not  surprising.  Knowledge  about  costs  gained  from  the 
development  phase  should  allow  for  better  estimation  of  costs 
during  production. 15 


SUMMARY  AND  CONCLUSIONS 

This  study  has  examined  relationships  between  extensions  in 
technology  and  both  development  cost  and  production  cost 
associated  with  achieving  extensions.  The  result  has  been  to 


15Average  (absolute)  variance  in  Figure  4  is  27%.  Model  2, 
Table  9  also  provides  a  good  prediction  of  production  cost. 
Using  Model  2  to  predict  production  cost  resulted  in  an  average 
variance  of  35%. 
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SATELLITE  PROGRAMS 


develop  cost  estimating  approaches  and  models  that  are  workable 
for  both  cost  prediction  and  cost  control. 

Our  first  step  was  to  identify  technoloay  determining 
characteristics.  Technical  expertise  was  necessary  to  develop 
relevant  characteristics.  Such  characteristics  should  be  goals 
of  the  development  process,  alterable  by  engineering  development 
decisions,  and  ascertainable  at  an  early  stage  in  the  development 
process.  Factor  analysis  was  used  to  reduce  the  number  of 
technology  characteristics  to  a  number  manageable  during 
subsequent  analysis. 

Through  a  regression  model,  the  factor  scores  were  used  to 
predict  the  year  in  which  systems  become  operational.  High 
explanatory  ability  of  this  model  provided  indirect  confirmation 
that  the  factor  scores  appropriately  reflected  relevant 
dimensions  of  technology. 

This  regression  was  used  to  define  a  year-of-technology  for 
individual  systems,  from  which  measures  of  the  state  of 
technology  (STAND) ,  the  extension  in  technology  beyond  the 
current  state  of  the  art  (ADVANCE)  and  the  extent  of  technology 
embodied  in  each  system  (REACH)  were  developed. 

The  purpose  of  creating  these  technology  measures  was  to 
facilitate  prediction  of  the  cost  of  developing  new  technological 
systems,  and  to  offer  a  methodology  for  controlling  such  costs. 
It  was  therefore  necessary  to  hypothesize  and  test  for 
associations  between  the  technical  complexity  of  the  development 
tack  and  the  level  of  activity  required  to  complete  the  task. 
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Two  variables,  STAND  and  ADVANCE,  reflecting  technological 
complexity,  were  successfully  used  to  predict  development  time. 
Development  time  was  found  to  be  positively  related  to  both  the 
state  of  technology  at  the  time  a  development  project  was 
undertaken,  and  the  extension  in  technology  required. 

Predicted  and  residual  development  time  were  used  to  predict 
development  cost.  Results  indicated  that  development  cost  is  not 
a  smooth  function  of  time;  when  completion  drags  on  beyond  the 
"natural"  project  time,  cost  increases  more  rapidly,  consistent 
with  time  pressure  influencing  costs. 

This  two  stage  approach  to  estimating  development  cost  was 
found  to  be  superior  to  a  direct  production  of  cost  from  the 
technology  variables,  indicating  that  development  time  is  an 
intervening  variable  between  technological  complexity  and  cost. 

The  two  stage  approach  also  permitted  the  creation  of 
variance  measures  useful  for  controlling  costs.  The  Variance  Due 
to  Time  reflected  the  portion  of  the  Total  Cost  Variance 
attributable  to  the  cost  consequences  of  time  delays.  The  Cost 
Control  Variance  indicated  the  quality  of  cost  control  for 
proj  ects . 

Measures  resulting  from  the  analysis  of  the  development 
phase  were  then  used  to  predict  production  cost.  Production 
costs  were  found  to  be  related  to  the  scope  of  the  program,  best 
captured  by  the  actual  ex  post  development  cost.  This  result 
indicated  that  both  the  time  and  cost  control  variances  during 
the  development  phase  contain  information  useful  in  explaining 
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subsequent  production  costs;  i.e.  cost  "overruns"  experienced 
during  development  tend  to  be  permanent  and  reflected  in  higher 
product  i  on  cost. . 

Lastly,  the  idea  of  a  development  "premium"  was  introduced 
to  explain  the  ratio  of  development  cost  to  production  cost. 
Findings  indicated  that  the  greater  the  technological  complexity 
of  the  system,  the  larger  development  costs  are  relative  to 
production  costs.  This  relationship  was  then  used  to  determine 
expected  production  cost,  a  benchmark  from  which  a  production 
cost  variance,  useful  in  control  of  production  cost,  could  be 
calculated. 

The  fundamentals  developed  in  this  study  provide  a  workable 
methodology  for  measuring  technological  complexity  and  advances 
in  technology  in  systems.  The  methods  have  also  been  shown  tc  be 
effective  in  relating  technology  measures  to  the  cost  of 
development  and  production  programs.  The  approach  results  in 
variance  measures  useful  in  the  difficult  task  of  cost  control. 

The  approach  was  tested  on  only  one  set  of  data.  However, 
measures  developed  at  each  stage  of  the  analysis  were  used  in 
subsequent  stages,  and  at  each  stage  hypotheses  about  the 
expected  relationships  between  variables  were  confirmed.  This 
provides  some  evidence  of  the  internal  consistency  and  logic  of 
the  approach.  Of  course  future  research  applying  similar  methods 
to  additional  data  sets  would  be  beneficial. 


28 


REFERENCES 


Alexander,  A.  J.,  and  J.  R.  Nelson,  "Measuring  Technological 
Change:  Aircraft  Turbine  Engines,"  Technological  Forecasting  and 
Social  Change.  5  (1973),  pp .  189-203  . 

Dodson,  E.  N.  ,  and  C.  A.  Graver,  "An  Approach  to  Quantitative 
Measurement  of  Advances  in  State  of  the  Art,"  Internal  Memorandum 
(Releasable)  IMR-997,  General  Research  Corporation  (Santa 
Barbara,  1969),  39  pgs . 

Dodson,  E.  N.  ,  "Technological  Change  and  Cost  Analysis  of  High- 
Technology  Systems,"  IEEE  Transactions  on  Engineering  Management 
(May  1977) ,  pp.  38-45. 

_ ,  "Measurement  of  State  of  the  Art  and  Technological 

Advance,"  Technological  Forecasting  and  Social  Change.  27  (1985), 
pp.  129-146. 

Gordon,  T.  J.,  and  T.  R.  Munson,  "A  Proposed  Convention  for 
Measuring  the  State  of  the  Art  of  Products  or  Processes," 
Technological  Forecasting  and  Social  Change.  20  (1981),  pp.  1-26. 

Greer,  W.  R. ,  A  Method  for  Estimating  and  Controlling  The  Cost  cf 
Extending  Technology.  Naval  Postgraduate  School  Technical  Report, 
#  N PS-54 -88-002 ,  (Monterey,  CA,  1988),  101  pgs. 

Harman.  II.  H.  ,  Modern  Factor  Analysis.  3rd  ed.  (University  of 
Chicago,  1976) . 

Kaplan,  R.  ,  Advanced  Management  Accounting.  (Prentice  Hall, 
1982) ,  pp.  97-105. 

Kerlinger,  F.  N.,  Foundations  of  Behavioral  Research.  2nd  ed. 
(New  York:  Holt,  Rinehard  and  Winston,  1973). 

Knight,  K.  F.,  "A  Functional  and  Structural  Measurement  of 
Technology,"  Technological  Forecasting  and  Social  Change.  27 
(1985) ,  pp.  107-127 . 

Lienhard,  J.  H.,  "The  Rate  of  Technological  Improvement  before 
and  after  the  1830s,"  Technology  and  Culture  (July  1979),  pp. 
515-530. 

Martino,  J.  P.  ,  "Measurement  of  Technology  Using  Tradeoff 
Surfaces,"  Technological  Forecasting  and  Social  Change.  27 
(1985) ,  pp.  147-160. 

Neter,  J.  and  W.  Wasserman,  Applied  Linear  Statistical  Models, 
(Irwin ,  1974 ) . 


29 


Timperlake,  E.,  W.  Vogt,  and  L.  Sullivan,  The  Tascfcrm-Air  Model . 
Analytic  Sciences  Corporation  Research  Report,  TR-1334-3 
(Arlington,  VA,  February  1980) . 

Womer,  N.  K.  ,  "Learning  Curves,  Production  Rate  and  Program 
Costs,"  Management  Science.  April  25,  1979,  pp.  312-319. 


Appendix  I 

VARIABLES  DESCRIBING  SATELLITE  TECHNOLOGY 

Attitude  Control  System  (ACS)  variables: 

ACS  1  --  Pointing  accuracy  (reciprocal). 

ACS2  —  Primary  stabilization  Method  (coded;  Spin  =  0; 

moment,  inertial,  dual  =  1). 

ACS3  —  Maneuverability  (coded:  no  =  0,  yes  =  1). 

Apogee  Kick  Motor  (AKM)  variables: 

AKM1  —  Specific  impulse. 

AKM2  --  Propellent  weight  /  Dry  weight. 

Communicat ions  variable: 

COMM  --  Potjot-  required. 

Electrical  power  systems  variables: 

EPS 1  --  Battery  Capacity. 

EPS2  —  Beginning  of  life  power  /  Array  Area. 

EPS 3  —  Array  Topology  (coded:  body  =  0,  fixed  =  1, 
movable  =  2 ) . 

Mission  or  environmental  variables: 

LIFE  --  Design  life. 

NHARD  —  Degree  of  nuclear  hardening. 

LAUNCH  —  Launch  method  (coded:  missile  =  0,  Shuttle, 
dual  =  1) . 

QUALS  —  Percent  of  components  that  must  be  of  "space" 
quality  instead  of  merely  "aircraft"  quality. 
APOGEE  —  A  function  of  maximum  orbital  distance  from  earth, 
design  life  and  quality. 

DESIGN  --  Design  life  adjusted  for  quality. 

Thermal  variable: 

THERM  —  Tolerable  temperature  range  (maximum  -  minimum) . 

Tracking  telemetry  control  variable: 

TTC  --  Autonomous  operating  days. 
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Appendix  II 


This  appendix  describes  the  factor  analysis  process  used  to 
compress  the  17  raw  technology  measures16  down  to  four  factor 
scores.  The  procedure  used  was  part  heuristic,  part  statistical. 
Principle  components  factor  analysis  with  varimax  orthogonal 
rotation  was  conducted  on  the  original  17  variables.  Variables 
with  large  (>.50)  negative  factor  loadings  on  the  first  factor 
analysis  run  were  eliminated  since  characteristics  had  to  be 
consistently  specified  so  that  increasing  values  corresponded  to 
greater  technical  difficulty.17  This  eliminated  DESIGN,  ACS3 
and  EPS2 .  QUALS  was  also  discarded  because  it  had  no  substantial 
positive  factor  loading  (no  loading  greater  than  or  equal  to 
.50). 18  The  remaining  13  variables  were  factored  again  and 
clustered  onto  four  factors  with  78.5%  of  the  variance  explained. 

At  this  point  the  objective  in  eliminating  variables  became 
to  maximize  the  percentage  of  variance  explained.19  In  addition 
to  substantial  loadings  on  their  principal  factors,  ACS1,  TTC  and 
LAUNCH  had  significant  loadings  in  other  columns.  Through  trying 

16Data  was  available  for  all  17  variables  for  each  satellite 
with  the  exception  of  variable  THERM  for  satellite  R.  The  mean 
value  of  THERM  was  used  for  satellite  R  in  the  subsequent  analysis. 

17See  Dodson  and  Graver  [1969]  for  a  further  discussion  of 
this  constraint. 

18For  an  excellent  summary  of  variable  selection  criteria 
and  "simple"  matrix  structure  objectives  see  Kerlinger  [1973,  pp . 
672-73]. 

19This  is  a  common  practical  expedient  in  factor  analysis. 
See  Harman  [  1976,  p.  185]. 
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various  combinations  it  was  found  that  by  eliminating  ACS1  and 
TTC  but  retaining  LAUNCH  the  variance  explained  reached  a  maximum 
of  81.7%.  (Of  course,  this  is  not  an  optimization  method.  In 
general,  no  optimization  methods  exist  for  factor  analysis. 
There  is  no  guarantee  that  a  superior  solution  could  not  be 
reached  by  some  other  sequence  of  steps.  The  authors  can  only 
report  the  steps  taken  in  this  particular  analysis.)  Factor 
loadings  and  eigenvalues  from  the  final  rotated  factor  matrix  are 
shown  in  Table  11. 

Factor  Interpretation 

The  clustering  of  the  variables  and  the  strong  loadings  lead 
quite  easily  to  conclusions  as  to  the  nature  of  the  four  factors 
that  describe  the  technology  embedded  in  a  satellite. 

Factor  1  can  be  labeled  MISSION.  To  achieve  a  particular 
mission,  design  requirements  must  be  specified  reflecting  the 
distance  from  earth  (APOGEE) ,  design  life  (LIFE)  and  launch  mode 
(LAUNCH) .  While  the  variable  COMM  is  actually  required  power  for 
communications  equipment,  it  should  follow  from  mission 
specifications . 

Factor  2  consists  of  two  apogee  kick  motor  variables.  Since 
the  apogee  kick  motor  is  used  to  achieve  a  particular  orbit,  we 
have  labeled  this  factor  ORBITAL. 

Factor  3  consists  of  three  variables  reflecting  ELECTRICAL 
POWER  technology.  Battery  capacity  (EPS1)  and  array  topology 
(EPS3)  are  directly  related  to  the  technical  sophistication  of 
the  electrical  system.  The  primary  stabilization  method  (ACS2) 
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Table  11 


ROTATED  FACTOR  MATRIX 


FACTOR  1  FACTOR  2  FACTOR  3 


APOGEE 

.9632 

.0337 

-.0891 

LIFE 

.  9599 

.  0328 

.  0015 

COMM 

.  8391 

.  1331 

.3663 

LAUNCH 

.7376 

.0485 

.4279 

AKM2 

.0869 

.9802 

-.0465 

AKM1 

.  0370 

.9611 

-.2015 

ACS2 

.  1668 

-. 0800 

.8055 

EPS  3 

.  3438 

.  0127 

.7822 

EPS  1 

-.1284 

-.2747 

.  5379 

THERM 

.  1635 

-.0866 

-.1575 

NHARD 

.  0917 

.2131 

.  1706 

EIGENVALUE 

4 . 006 

2 . 348 

1.388 

FACTOR  4 

.  1204 
.  0456 
.  0616 
.4189 

.  0820 
.0516 

-.3237 
.2536 
.  0722 

.8464 

.7602 


1.250 


is  indirectly  associated  with  electrical  power  because  the 
stabilization  method  has  implications  for  array  deployment  which 
depends  on  array  topology  type. 

Factor  4  consists  of  two  variables  reflecting  the 
ENVIRONMENT  in  which  the  mission  is  to  be  conducted.  The 
temperature  range  (THERM)  affects  component  design,  while  the 
degree  of  nuclear  hardening  (NHARD)  is  directly  mission- 
determined. 
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